Social Thermoregulation Meta-analysis - Supplementary analysis outputs
Ivan Ropovik
2021-02-04
This is the supplementary analytic output for the paper Social Thermoregulation: A Meta-analysis by IJzerman et al.
It reports detailed results for all models reported in the paper. The analytic R script by which this html report was generated can be found on the project’s OSF page at: [LINK].
Brief information about the methods used in the analysis:
RMA results with model-based SEs k = number of studies; sqrt in “Variance components” = tau, the standard deviation of true effects; estimate in “Model results” = naive MA estimate
RVE SEs with Satterthwaite small-sample correction Estimate based on a multilevel RE model with constant sampling correlation model (CHE - correlated hierarchical effects - working model) (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Interpretation of naive-meta-analysis should be based on these estimates.
Prediction interval Shows the expected range of true effects in similar studies. As an approximation, in 95% of cases the true effect in a new published study can be expected to fall between PI LB and PI UB. Note that these are non-adjusted estimates. An unbiased newly conducted study will more likely fall in an interval centered around bias-adjusted estimate with a wider CI width.
Heterogeneity Tau can be interpreted as the total amount of heterogeneity in the true effects. I^2$ represents the ratio of true heterogeneity to total variance across the observed effect estimates. Estimates calculated by two approaches are reported. This is followed by separate estimates of between- and within-cluster heterogeneity and estimated intra-class correlation of underlying true effects.
Proportion of significant results What proportion of effects were statistically at the alpha level of .05.
ES-precision correlation Kendalls’s correlation between the ES and precision.
4/3PSM Applies a permutation-based, step-function 4-parameter selection model (one-tailed p-value steps = c(.025, .5, 1)). Falls back to 3-parameter selection model if at least one of the three p-value intervals contains less than 5 p-values. For this meta-analysis, we applied 3-parameter selection model by default as there were only 11 independent effects in the opposite direction overall (6%), causing the estimates to be unstable across iterations. pvalue = p-value testing H0 that the effect is zero. ciLB and ciUB are lower and upper bound of the CI. k = number of studies. steps = 3 means that the 4PSM was applied, 2 means that the 3PSM was applied.
PET-PEESE Estimated effect size of an infinitely precise study. Using 4/3PSM as the conditional estimator instead of PET (can be changed to PET). If the PET-PEESE estimate is in the opposite direction, the effect can be regarded nil. By default (can be changed to PET), the function employs a modified sample-size based estimator (see https://www.jepusto.com/pet-peese-performance/). It also uses the same RVE sandwich-type based estimator in a CHE (correlated hierarchical effects) working model with the identical random effects structure as the primary (naive) meta-analytic model.
We report results for both, PET and PEESE, with the first reported one being the primary (based on the conditional estimator).
WAAP-WLS The combined WAAP-WLS estimator (weighted average of the adequately powered - weighted least squares) tries to identify studies that are adequately powered to detect the meta-analytic effect. If there is less than two such studies, the method falls back to the WLS estimator (Stanley & Doucouliagos, 2015). If there are at least two adequately powered studies, WAAP returns a WLS estimate based on effects from only those studies.
type = 1: WAAP estimate, 2: WLS estimate. kAdequate = number of adequately powered studies
p-uniform P-uniform* is a selection model conceptually similar to p-curve. It makes use of the fact that p-values follow a uniform distribution at the true effect size while it includes also nonsignificant effect sizes. Permutation-based new version of p-uniform method, the so-called p-uniform* (van Aert, van Assen, 2021).
p-curve Permutation-based p-curve method. Output should be self-explanatory. For more info see p-curve.com
Power for detecting SESOI and bias-corrected parameter estimates Estimates of the statistical power for detecting a smallest effect sizes of interest equal to .20, .50, and .70 in SD units (Cohen’s d). A sort of a thought experiment, we also assumed that population true values equal the bias-corrected estimates (4/3PSM or PET-PEESE) and computed power for those.
Handling of dependencies in bias-correction methods To handle dependencies among the effects, the 4PSM, p-curve, p-uniform are implemented using a permutation-based procedure, randomly selecting only one focal effect (i.e., excluding those which were not coded as being focal) from a single study and iterating nIterations times. Lastly, the procedure selects the result with the median value of the ES estimate (4PSM, p-uniform) or median z-score of the full p-curve (p-curve).
Descriptives
Sample sizes
N of effects
## [1] 192N of studies
## [1] 189N of papers
## [1] 76N of effects eligible just for evidential value test
## [1] 91Median N across all the ES eligible for meta-analysis
## [1] 69Meta-analysis results
Moderation by prior experiences in relationships
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 19; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0177 0.1329 9 no study
## sigma^2.2 0.1215 0.3486 19 no study/result
##
## Test for Heterogeneity:
## Q(df = 18) = 107.2633, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.5964 0.1208 4.9371 <.0001 0.3596 0.8332 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.596 0.12 4.96 6.46 0.00207 **
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.596 0.12 6.46 0.307 0.886
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.307 1.500
##
## $Heterogeneity
## Tau I^2
## 0.3731153 79.3790303
## Jackson's I^2 Between-cluster heterogeneity
## 87.9600000 10.0700000
## Within-cluster heterogeneity ICC
## 69.3000000 0.1300000
##
## $`Proportion of significant results`
## [1] 0.7894737
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.2286703
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.340 0.265 1.284 0.199 -0.179 0.859 9.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## 0.177 0.431 0.411 0.694 -0.842
## ciUB PEESE estimate se zvalue pvalue
## 1.196 0.378 0.257 1.472 0.185
## ciLB ciUB
## -0.229 0.986
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.5006211 0.1378544 0.1378544 0.006670333 0.1827282 0.8185139
## type kAdequate
## 1 1 9
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.51375198 0.07827554 0.89937291 0.12258993
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 9
## - Total number of p<0.05 studies included into the analysis: k = 7 (77.78%)
## - Total number of studies with p<0.025: k = 6 (66.67%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.062 -7.249 0 -7.418 0
## Flatness test 0.905 5.068 1 6.907 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 98% (90.8%-99%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.227
## Median power for detecting a SESOI of d = .50
## 0.856
## Median power for detecting a SESOI of d = .70
## 0.988
## Median power for detecting PET-PEESE estimate.PET estimate
## 0.188
## Median power for detecting 4/3PSM estimate.est
## 0.538Forest plot and Contour-enhanced funnel plot
P-curve plot
Effect type
## [1] "The compensatory vs priming effects conceptualized by the actual direction of the effect as contrast vs. assimilation"Compensatory
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 45; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0348 0.1866 35 no study
## sigma^2.2 0.0073 0.0855 45 no study/result
##
## Test for Heterogeneity:
## Q(df = 44) = 132.7217, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.3311 0.0446 7.4258 <.0001 0.2437 0.4185 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.331 0.0446 7.43 30.7 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.331 0.0446 30.7 0.24 0.422
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.096 0.758
##
## $Heterogeneity
## Tau I^2
## 0.2052501 71.6443532
## Jackson's I^2 Between-cluster heterogeneity
## 83.2400000 59.2000000
## Within-cluster heterogeneity ICC
## 12.4400000 0.8300000
##
## $`Proportion of significant results`
## [1] 0.5106383
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.58586
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.263 0.069 3.841 0.000 0.129 0.397 35.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.214 0.067 3.200 0.003 0.078
## ciUB PET estimate se zvalue pvalue
## 0.351 0.038 0.117 0.326 0.747
## ciLB ciUB
## -0.200 0.276
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.2408241 0.1099448 0.1099448 0.1162097 -0.1090692 0.5907174
## type kAdequate
## 1 1 4
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.2776763154 0.1450814742 0.4067088443 0.0005412508
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 52
## - Total number of p<0.05 studies included into the analysis: k = 36 (69.23%)
## - Total number of studies with p<0.025: k = 25 (48.08%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.014 -6.934 0.000 -6.997 0
## Flatness test 0.459 3.037 0.999 9.525 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 68% (49.9%-81.1%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.385
## Median power for detecting a SESOI of d = .50
## 0.986
## Median power for detecting a SESOI of d = .70
## 1.000
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.430
## Median power for detecting 4/3PSM estimate.est
## 0.592Priming
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 125; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0347 0.1864 84 no study
## sigma^2.2 0.0196 0.1399 125 no study/result
##
## Test for Heterogeneity:
## Q(df = 124) = 315.2924, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4497 0.0339 13.2675 <.0001 0.3833 0.5162 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.45 0.0339 13.3 73.9 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.45 0.0339 73.9 0.382 0.517
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.019 0.918
##
## $Heterogeneity
## Tau I^2
## 0.2330976 60.9134712
## Jackson's I^2 Between-cluster heterogeneity
## 76.8600000 38.9600000
## Within-cluster heterogeneity ICC
## 21.9600000 0.6400000
##
## $`Proportion of significant results`
## [1] 0.6136364
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5434097
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.248 0.052 4.780 0.000 0.147 0.350 83.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.204 0.056 3.656 0.000 0.093
## ciUB PET estimate se zvalue pvalue
## 0.315 -0.049 0.060 -0.810 0.421
## ciLB ciUB
## -0.169 0.071
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low
## 1 WAAP-WLS b0 0.06033676 0.03493944 0.03493944 0.1349326 -0.02515696
## conf.high type kAdequate
## 1 0.1458305 1 7
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.28237593329 0.17999288439 0.38133351331 0.00001748352
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 116
## - Total number of p<0.05 studies included into the analysis: k = 80 (68.97%)
## - Total number of studies with p<0.025: k = 50 (43.1%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.016 -6.251 0.000 -8.037 0
## Flatness test 0.053 0.347 0.636 9.925 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 36% (23.4%-49.7%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.202
## Median power for detecting a SESOI of d = .50
## 0.801
## Median power for detecting a SESOI of d = .70
## 0.976
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.209
## Median power for detecting 4/3PSM estimate.est
## 0.286Table 1
## k g [95% CI] SE Tau I^2 3PSM est [95% CI] 3PSM.pvalue
## Compensatory 47 0.33 [0.24, 0.42] 0.04 0.21 72% 0.26 [0.13, 0.4] 0
## Priming 132 0.45 [0.38, 0.52] 0.03 0.23 61% 0.25 [0.15, 0.35] 0
## PET-PEESE est [95% CI] PET-PEESE.pvalue
## Compensatory 0.21 [0.08, 0.35] 0.003
## Priming 0.2 [0.09, 0.32] 0Comparison of effect types
Model without covariates
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 factor(effectCompPriming)1 0.345 0.0461 7.48 <0.001 ***
## 2 factor(effectCompPriming)2 0.443 0.0324 13.65 <0.001 ***
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(effectCompPriming)1 0.345 0.0461 Inf 0.254 0.435
## 2 factor(effectCompPriming)2 0.443 0.0324 Inf 0.379 0.506
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 2.81 1 47.5 0.1Model with covariates
Controlling for design-related factors that are prognostic w.r.t. the effect sizes (i.e., might vary across moderator categories), namely rct, published, sourceTargetDirectionality, and studentSample.
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 factor(effectCompPriming)1 0.0932 0.0721 1.293 0.19600
## 2 factor(effectCompPriming)2 0.1769 0.0898 1.969 0.04896 *
## 3 rct 0.1463 0.0564 2.595 0.00945 **
## 4 published 0.1285 0.0630 2.040 0.04138 *
## 5 sourceTargetDirectionality_reconcil -0.0305 0.0681 -0.448 0.65428
## 6 studentSample 0.0789 0.0684 1.153 0.24873
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI
## 1 factor(effectCompPriming)1 0.0932 0.0721 Inf -0.048070
## 2 factor(effectCompPriming)2 0.1769 0.0898 Inf 0.000805
## 3 rct 0.1463 0.0564 Inf 0.035802
## 4 published 0.1285 0.0630 Inf 0.005026
## 5 sourceTargetDirectionality_reconcil -0.0305 0.0681 Inf -0.163877
## 6 studentSample 0.0789 0.0684 Inf -0.055188
## Upper 95% CI
## 1 0.234
## 2 0.353
## 3 0.257
## 4 0.252
## 5 0.103
## 6 0.213
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 2.13 1 26.1 0.156Plots
Forest plots
Contour-enhanced funnel plots
p-curve plots
PET-PEESE plots
top = Compensatory, bottom = Priming
Using the sqrt(2/n) and 2/n terms instead of SE and var for PET and PEESE, respectively since modified sample-size based estimator was implemented (see https://www.jepusto.com/pet-peese-performance/).
Mood
Results
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 17; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 11 no study
## sigma^2.2 0.0000 0.0000 17 no study/result
##
## Test for Heterogeneity:
## Q(df = 16) = 7.1492, p-val = 0.9703
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2315 0.0698 3.3145 0.0009 0.0946 0.3684 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.231 0.0448 5.17 8.9 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.231 0.0448 8.9 0.13 0.333
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.131 0.332
##
## $Heterogeneity
## Tau I^2
## 0.00000295647900 0.00000001354476
## Jackson's I^2 Between-cluster heterogeneity
## 0.00000000000000 0.00000000000000
## Within-cluster heterogeneity ICC
## 0.00000000000000 1.00000000000000
##
## $`Proportion of significant results`
## [1] 0.1176471
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Forest plot and Contour-enhanced funnel plot
Overall effect results
Results
Number of iterations run equal to 200 for p-curve and 5000 for all other bias correction functions.
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 175; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0438 0.2092 116 no study
## sigma^2.2 0.0357 0.1888 175 no study/result
##
## Test for Heterogeneity:
## Q(df = 174) = 677.7377, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4085 0.0315 12.9864 <.0001 0.3469 0.4702 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.409 0.0315 13 105 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.409 0.0315 105 0.346 0.471
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.153 0.970
##
## $Heterogeneity
## Tau I^2
## 0.2818135 74.9082576
## Jackson's I^2 Between-cluster heterogeneity
## 85.1400000 41.2700000
## Within-cluster heterogeneity ICC
## 33.6400000 0.5500000
##
## $`Proportion of significant results`
## [1] 0.5923913
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5796001
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.187 0.052 3.599 0.000 0.085 0.289 114.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.159 0.053 3.008 0.003 0.054
## ciUB PET estimate se zvalue pvalue
## 0.264 -0.138 0.072 -1.915 0.058
## ciLB ciUB
## -0.280 0.005
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.1493071 0.09129957 0.09129957 0.2436033 -0.2435232 0.5421375
## type kAdequate
## 1 1 3
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.1944982257 0.0957078307 0.2920326952 0.0005655032
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 166
## - Total number of p<0.05 studies included into the analysis: k = 115 (69.28%)
## - Total number of studies with p<0.025: k = 75 (45.18%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.001 -8.782 0.000 -10.217 0
## Flatness test 0.087 1.653 0.951 13.089 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 45% (33.4%-55.5%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.215
## Median power for detecting a SESOI of d = .50
## 0.830
## Median power for detecting a SESOI of d = .70
## 0.983
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.153
## Median power for detecting 4/3PSM estimate.est
## 0.193Forest plot
Contour-enhanced funnel plot
p-curve plot
Methods
Moderator analysis
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 methodPhysical temperature manipulation 0.464 0.0629 7.37 < 0.001 ***
## 2 methodVisual/verbal temperature prime 0.485 0.0589 8.23 < 0.001 ***
## 3 methodOutside temperature 0.379 0.1512 2.51 0.01223 *
## 4 methodTemperature estimate as DV 0.477 0.0703 6.79 < 0.001 ***
## 5 methodSubjective warmth judgment as DV 0.299 0.1149 2.61 0.00915 **
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI
## 1 methodPhysical temperature manipulation 0.464 0.0629 Inf 0.3406
## 2 methodVisual/verbal temperature prime 0.485 0.0589 Inf 0.3693
## 3 methodOutside temperature 0.379 0.1512 Inf 0.0825
## 4 methodTemperature estimate as DV 0.477 0.0703 Inf 0.3396
## 5 methodSubjective warmth judgment as DV 0.299 0.1149 Inf 0.0743
## Upper 95% CI
## 1 0.587
## 2 0.600
## 3 0.675
## 4 0.615
## 5 0.525
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 0.682 4 17.5 0.614Subgroup analysis
Leaving out the Core temperature measurement and Skin temperature measurement, since k is too low.
Meta-analysis results
Brief results
## k g [95% CI] SE Tau I^2
## Physical.temperature.manipulation 83 0.48 [0.36, 0.59] 0.06 0.34 69%
## Visual.verbal.temperature.prime 23 0.44 [0.35, 0.53] 0.04 0.15 38%
## Outside.temperature 13 0.16 [-0.02, 0.34] 0.06 0.1 44%
## Temperature.estimate.as.DV 25 0.39 [0.27, 0.52] 0.06 0.19 60%
## Subjective.warmth.judgment.as.DV 14 0.28 [0.03, 0.53] 0.11 0.37 93%Physical temperature manipulation
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 82; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.1003 0.3167 51 no study
## sigma^2.2 0.0153 0.1238 82 no study/result
##
## Test for Heterogeneity:
## Q(df = 81) = 339.5264, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4779 0.0575 8.3075 <.0001 0.3652 0.5907 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.478 0.0576 8.3 48.2 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.478 0.0576 48.2 0.362 0.594
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.215 1.171
##
## $Heterogeneity
## Tau I^2
## 0.3400426 69.0555528
## Jackson's I^2 Between-cluster heterogeneity
## 80.9300000 59.9000000
## Within-cluster heterogeneity ICC
## 9.1500000 0.8700000
##
## $`Proportion of significant results`
## [1] 0.6746988
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Visual/verbal temperature prime
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 23; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 14 no study
## sigma^2.2 0.0232 0.1524 23 no study/result
##
## Test for Heterogeneity:
## Q(df = 22) = 38.5260, p-val = 0.0160
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4398 0.0600 7.3355 <.0001 0.3223 0.5573 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.44 0.0411 10.7 11.4 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.44 0.0411 11.4 0.35 0.53
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.099 0.781
##
## $Heterogeneity
## Tau I^2
## 0.1523991 37.8013636
## Jackson's I^2 Between-cluster heterogeneity
## 13.6800000 0.0000000
## Within-cluster heterogeneity ICC
## 37.8000000 0.0000000
##
## $`Proportion of significant results`
## [1] 0.6956522
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Outside temperature
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 8; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0077 0.0875 6 no study
## sigma^2.2 0.0028 0.0525 8 no study/result
##
## Test for Heterogeneity:
## Q(df = 7) = 14.2050, p-val = 0.0477
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.1607 0.0642 2.5034 0.0123 0.0349 0.2864 *
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.161 0.0634 2.54 3.63 0.0705 .
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.161 0.0634 3.63 -0.0225 0.344
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.147 0.468
##
## $Heterogeneity
## Tau I^2
## 0.1020439 44.1844740
## Jackson's I^2 Between-cluster heterogeneity
## 56.9100000 32.5100000
## Within-cluster heterogeneity ICC
## 11.6700000 0.7400000
##
## $`Proportion of significant results`
## [1] 0.1538462
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Temperature estimate as DV
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 23; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0378 0.1945 21 no study
## sigma^2.2 0.0000 0.0000 23 no study/result
##
## Test for Heterogeneity:
## Q(df = 22) = 49.5679, p-val = 0.0007
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.3931 0.0599 6.5631 <.0001 0.2757 0.5104 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.393 0.0601 6.54 17 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.393 0.0601 17 0.266 0.52
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.031 0.817
##
## $Heterogeneity
## Tau I^2
## 0.1945094 59.9035594
## Jackson's I^2 Between-cluster heterogeneity
## 71.0500000 59.9000000
## Within-cluster heterogeneity ICC
## 0.0000000 1.0000000
##
## $`Proportion of significant results`
## [1] 0.72
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Subjective warmth judgment as DV
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 13; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.1253 0.3539 12 no study
## sigma^2.2 0.0089 0.0942 13 no study/result
##
## Test for Heterogeneity:
## Q(df = 12) = 69.2488, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2810 0.1153 2.4362 0.0148 0.0549 0.5070 *
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.281 0.115 2.45 10.7 0.033 *
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.281 0.115 10.7 0.0274 0.535
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.564 1.126
##
## $Heterogeneity
## Tau I^2
## 0.3662546 92.6858591
## Jackson's I^2 Between-cluster heterogeneity
## 95.7600000 86.5600000
## Within-cluster heterogeneity ICC
## 6.1300000 0.9300000
##
## $`Proportion of significant results`
## [1] 0.5714286
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Bias correction results
Physical temperature manipulation
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5226101
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.151 0.094 1.603 0.109 -0.034 0.336 51.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.254 0.225 -1.128 0.265 -0.705
## ciUB PEESE estimate se zvalue pvalue
## 0.198 0.190 0.133 1.424 0.161
## ciLB ciUB
## -0.078 0.457
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 -0.4364618 0.251926 0.251926 0.2253231 -1.520412 0.6474885
## type kAdequate
## 1 1 3
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.06957033 -0.13193200 0.27026087 0.52185334
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 82
## - Total number of p<0.05 studies included into the analysis: k = 58 (70.73%)
## - Total number of studies with p<0.025: k = 33 (40.24%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.179 -6.720 0.000 -8.942 0
## Flatness test 0.013 1.791 0.963 10.255 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 50% (34.7%-65%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.19"
## Median power for detecting a SESOI of d = .50
## "0.768"
## Median power for detecting a SESOI of d = .70
## "0.965"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.129"Visual/verbal temperature prime
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.3287924
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.271 0.080 3.400 0.001 0.115 0.428 14.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.292 0.110 2.662 0.021 0.053
## ciUB PET estimate se zvalue pvalue
## 0.532 0.182 0.207 0.878 0.397
## ciLB ciUB
## -0.270 0.634
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.3037822 0.03003057 0.03003057 0.009631472 0.1745711 0.4329934
## type kAdequate
## 1 1 3
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.31186560 0.09457384 0.49685638 0.23877258
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 23
## - Total number of p<0.05 studies included into the analysis: k = 19 (82.61%)
## - Total number of studies with p<0.025: k = 14 (60.87%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.032 -4.527 0.000 -4.793 0
## Flatness test 0.670 1.450 0.926 5.666 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 56% (30.3%-77.3%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.202
## Median power for detecting a SESOI of d = .50
## 0.801
## Median power for detecting a SESOI of d = .70
## 0.976
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.374
## Median power for detecting 4/3PSM estimate.est
## 0.331Temperature estimate as DV
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.7500059
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.046 0.058 0.779 0.436 -0.069 0.160 21.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.086 0.091 -0.942 0.358 -0.277
## ciUB PEESE estimate se zvalue pvalue
## 0.105 0.108 0.062 1.742 0.098
## ciLB ciUB
## -0.022 0.238
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.04964282 0.1242567 0.1242567 0.7580266 -1.529189 1.628474
## type kAdequate
## 1 1 2
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.0556670 -0.1507642 0.2412312 0.6814799
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 24
## - Total number of p<0.05 studies included into the analysis: k = 18 (75%)
## - Total number of studies with p<0.025: k = 10 (41.67%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.407 -0.951 0.171 -1.939 0.026
## Flatness test 0.112 -1.590 0.056 3.610 1.000
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 10% (5%-34.4%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.202"
## Median power for detecting a SESOI of d = .50
## "0.801"
## Median power for detecting a SESOI of d = .70
## "0.976"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.058"Contour-enhanced funnel plots
Forest plots
Categories
Leaving out the Robotics and Neural Mechanisms, since k is too low
Results for different categories
Meta-analysis results
Brief results
## k g [95% CI] SE Tau I^2
## Emotion 24 0.39 [0.31, 0.46] 0.03 0.15 54%
## Interpersonal 76 0.36 [0.26, 0.46] 0.05 0.31 78%
## Person.perception 39 0.41 [0.23, 0.58] 0.08 0.33 81%
## Group.processes 12 0.62 [0.38, 0.85] 0.09 0 0%
## Moral.judgment 6 0.49 [-0.12, 1.1] 0.11 0.03 2%
## Self.regulation 26 0.32 [0.17, 0.47] 0.07 0.27 76%
## Cognitive.processes 36 0.56 [0.46, 0.66] 0.05 0.12 20%
## Economic.decision.making 43 0.44 [0.28, 0.59] 0.07 0.31 70%Emotion
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 23; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 19 no study
## sigma^2.2 0.0233 0.1526 23 no study/result
##
## Test for Heterogeneity:
## Q(df = 22) = 44.6519, p-val = 0.0029
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.3859 0.0506 7.6308 <.0001 0.2868 0.4850 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.386 0.0343 11.2 15.3 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.386 0.0343 15.3 0.313 0.459
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.057 0.715
##
## $Heterogeneity
## Tau I^2
## 0.1526369 54.0804386
## Jackson's I^2 Between-cluster heterogeneity
## 65.7500000 0.0000000
## Within-cluster heterogeneity ICC
## 54.0800000 0.0000000
##
## $`Proportion of significant results`
## [1] 0.7083333
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Interpersonal
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 75; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0503 0.2242 56 no study
## sigma^2.2 0.0488 0.2208 75 no study/result
##
## Test for Heterogeneity:
## Q(df = 74) = 364.3079, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.3594 0.0498 7.2141 <.0001 0.2617 0.4570 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.359 0.0499 7.21 51.7 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.359 0.0499 51.7 0.259 0.459
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.279 0.998
##
## $Heterogeneity
## Tau I^2
## 0.3146483 77.6948925
## Jackson's I^2 Between-cluster heterogeneity
## 85.2200000 39.4400000
## Within-cluster heterogeneity ICC
## 38.2600000 0.5100000
##
## $`Proportion of significant results`
## [1] 0.5789474
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Person perception
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 36; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0765 0.2766 21 no study
## sigma^2.2 0.0337 0.1835 36 no study/result
##
## Test for Heterogeneity:
## Q(df = 35) = 133.8868, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4075 0.0829 4.9175 <.0001 0.2451 0.5698 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.407 0.0828 4.92 18.3 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.407 0.0828 18.3 0.234 0.581
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.306 1.121
##
## $Heterogeneity
## Tau I^2
## 0.3319022 81.0462124
## Jackson's I^2 Between-cluster heterogeneity
## 90.0500000 56.2800000
## Within-cluster heterogeneity ICC
## 24.7700000 0.6900000
##
## $`Proportion of significant results`
## [1] 0.4358974
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Group processes
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 11; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 7 no study
## sigma^2.2 0.0000 0.0000 11 no study/result
##
## Test for Heterogeneity:
## Q(df = 10) = 8.8815, p-val = 0.5434
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.6152 0.1079 5.7024 <.0001 0.4037 0.8266 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.615 0.094 6.55 5.3 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.615 0.094 5.3 0.378 0.853
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.386 0.845
##
## $Heterogeneity
## Tau I^2
## 0.00000462993551 0.00000002010181
## Jackson's I^2 Between-cluster heterogeneity
## 0.00000000000000 0.00000000000000
## Within-cluster heterogeneity ICC
## 0.00000000000000 0.83000000000000
##
## $`Proportion of significant results`
## [1] 0.8333333
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Moral judgment
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 6; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 3 no study
## sigma^2.2 0.0011 0.0329 6 no study/result
##
## Test for Heterogeneity:
## Q(df = 5) = 4.6596, p-val = 0.4588
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4878 0.1076 4.5335 <.0001 0.2769 0.6987 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.488 0.111 4.41 1.61 0.0701 .
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.488 0.111 1.61 -0.12 1.1
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.027 0.948
##
## $Heterogeneity
## Tau I^2
## 0.03292223 2.34692161
## Jackson's I^2 Between-cluster heterogeneity
## 25.18000000 0.00000000
## Within-cluster heterogeneity ICC
## 2.35000000 0.00000000
##
## $`Proportion of significant results`
## [1] NA
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Self-regulation
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 24; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0312 0.1765 21 no study
## sigma^2.2 0.0436 0.2089 24 no study/result
##
## Test for Heterogeneity:
## Q(df = 23) = 90.2528, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.3242 0.0716 4.5298 <.0001 0.1839 0.4645 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.324 0.0714 4.54 18.7 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.324 0.0714 18.7 0.175 0.474
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.265 0.914
##
## $Heterogeneity
## Tau I^2
## 0.2734749 76.4112635
## Jackson's I^2 Between-cluster heterogeneity
## 84.2000000 31.8400000
## Within-cluster heterogeneity ICC
## 44.5800000 0.4200000
##
## $`Proportion of significant results`
## [1] 0.5384615
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Cognitive processes
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 35; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 29 no study
## sigma^2.2 0.0143 0.1194 35 no study/result
##
## Test for Heterogeneity:
## Q(df = 34) = 40.8129, p-val = 0.1959
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.5566 0.0485 11.4785 <.0001 0.4616 0.6517 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.557 0.0478 11.6 23.2 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.557 0.0478 23.2 0.458 0.655
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.293 0.820
##
## $Heterogeneity
## Tau I^2
## 0.1194099 19.7930556
## Jackson's I^2 Between-cluster heterogeneity
## 26.6100000 0.0000000
## Within-cluster heterogeneity ICC
## 19.7900000 0.0000000
##
## $`Proportion of significant results`
## [1] 0.75
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Economic decision-making
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 38; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0671 0.2590 25 no study
## sigma^2.2 0.0306 0.1750 38 no study/result
##
## Test for Heterogeneity:
## Q(df = 37) = 117.7860, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4361 0.0743 5.8697 <.0001 0.2905 0.5818 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.436 0.0743 5.87 22.7 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.436 0.0743 22.7 0.282 0.59
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.227 1.099
##
## $Heterogeneity
## Tau I^2
## 0.312574 70.182750
## Jackson's I^2 Between-cluster heterogeneity
## 81.650000 48.180000
## Within-cluster heterogeneity ICC
## 22.000000 0.690000
##
## $`Proportion of significant results`
## [1] 0.5581395
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Bias correction results
Emotion
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.2669407
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.260 0.028 9.171 0.000 0.204 0.315 19.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.353 0.050 7.110 0.000 0.249
## ciUB PET estimate se zvalue pvalue
## 0.458 0.301 0.064 4.724 0.000
## ciLB ciUB
## 0.167 0.436
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low
## 1 WAAP-WLS b0 0.3105771 0.02925429 0.02925429 0.000000000402088 0.2499074
## conf.high type kAdequate
## 1 0.3712468 2 1
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.30896207 0.19689270 0.41491368 0.01497127
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 31
## - Total number of p<0.05 studies included into the analysis: k = 23 (74.19%)
## - Total number of studies with p<0.025: k = 14 (45.16%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.202 -4.994 0.000 -6.551 0
## Flatness test 0.185 1.740 0.959 7.135 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 59% (34.7%-78%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.331
## Median power for detecting a SESOI of d = .50
## 0.968
## Median power for detecting a SESOI of d = .70
## 1.000
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.767
## Median power for detecting 4/3PSM estimate.est
## 0.508Interpersonal
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5905207
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.100 0.078 1.295 0.195 -0.052 0.252 56.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.215 0.107 -2.000 0.051 -0.430
## ciUB PEESE estimate se zvalue pvalue
## 0.000 0.098 0.077 1.276 0.207
## ciLB ciUB
## -0.056 0.252
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low
## 1 WAAP-WLS b0 0.1388678 0.04068096 0.04068096 0.001043176 0.05780919
## conf.high type kAdequate
## 1 0.2199264 2 1
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.07458773 -0.08100454 0.23262782 0.35226835
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 87
## - Total number of p<0.05 studies included into the analysis: k = 56 (64.37%)
## - Total number of studies with p<0.025: k = 31 (35.63%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.252 -4.513 0.000 -6.905 0
## Flatness test 0.008 -0.084 0.466 8.829 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 32% (17.9%-49.2%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.205"
## Median power for detecting a SESOI of d = .50
## "0.808"
## Median power for detecting a SESOI of d = .70
## "0.977"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.087"Person perception
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6107588
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.442 0.125 3.528 0.000 0.196 0.687 21.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.060 0.050 1.181 0.252 -0.046
## ciUB PET estimate se zvalue pvalue
## 0.165 -0.236 0.064 -3.710 0.001
## ciLB ciUB
## -0.369 -0.103
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low
## 1 WAAP-WLS b0 0.1963766 0.05084897 0.05084897 0.0004645318 0.09314768
## conf.high type kAdequate
## 1 0.2996055 2 0
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.3497897 0.1311900 0.5620256 0.0055566
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 27
## - Total number of p<0.05 studies included into the analysis: k = 18 (66.67%)
## - Total number of studies with p<0.025: k = 12 (44.44%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.119 -5.415 0.000 -6.263 0
## Flatness test 0.413 2.363 0.991 6.659 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 69% (44.9%-85.8%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.253
## Median power for detecting a SESOI of d = .50
## 0.899
## Median power for detecting a SESOI of d = .70
## 0.995
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.067
## Median power for detecting 4/3PSM estimate.est
## 0.816Self-regulation
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6350534
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.068 0.114 0.600 0.548 -0.155 0.292 21.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.318 0.101 -3.148 0.005 -0.530
## ciUB PEESE estimate se zvalue pvalue
## -0.107 -0.003 0.056 -0.057 0.955
## ciLB ciUB
## -0.120 0.114
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.1496 0.05771728 0.05771728 0.01630102 0.03020272 0.2689973
## type kAdequate
## 1 2 1
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.12567927 -0.05613884 0.31311091 0.18274946
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 35
## - Total number of p<0.05 studies included into the analysis: k = 24 (68.57%)
## - Total number of studies with p<0.025: k = 15 (42.86%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.154 -2.474 0.007 -2.071 0.019
## Flatness test 0.225 -0.642 0.260 4.357 1.000
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 24% (8.1%-48.8%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.396"
## Median power for detecting a SESOI of d = .50
## "0.989"
## Median power for detecting a SESOI of d = .70
## "1"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.089"Cognitive processes
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.3800724
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.344 0.129 2.678 0.007 0.092 0.596 28.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.405 0.083 4.859 0.000 0.234
## ciUB PET estimate se zvalue pvalue
## 0.576 0.218 0.146 1.490 0.148
## ciLB ciUB
## -0.082 0.519
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.4210579 0.1142019 0.1142019 0.02107484 0.1039825 0.7381332
## type kAdequate
## 1 1 5
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.24115193 0.01326872 0.44907076 0.21851476
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 51
## - Total number of p<0.05 studies included into the analysis: k = 38 (74.51%)
## - Total number of studies with p<0.025: k = 25 (49.02%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.036 -5.536 0.00 -6.390 0
## Flatness test 0.273 1.403 0.92 7.798 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 50% (30.6%-67.2%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.197
## Median power for detecting a SESOI of d = .50
## 0.789
## Median power for detecting a SESOI of d = .70
## 0.972
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.609
## Median power for detecting 4/3PSM estimate.est
## 0.476Economic decision-making
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6508614
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.234 0.125 1.875 0.061 -0.011 0.478 24.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.241 0.134 1.792 0.086 -0.037
## ciUB PET estimate se zvalue pvalue
## 0.519 -0.135 0.257 -0.526 0.604
## ciLB ciUB
## -0.668 0.397
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low
## 1 WAAP-WLS b0 0.3017689 0.05936583 0.05936583 0.00001090482 0.1814823
## conf.high type kAdequate
## 1 0.4220555 2 0
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.229290001 -0.009305462 0.461452354 0.089691375
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 36
## - Total number of p<0.05 studies included into the analysis: k = 27 (75%)
## - Total number of studies with p<0.025: k = 14 (38.89%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.500 -2.332 0.010 -3.352 0
## Flatness test 0.024 -0.937 0.174 4.609 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 21% (7.3%-43.7%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.202
## Median power for detecting a SESOI of d = .50
## 0.801
## Median power for detecting a SESOI of d = .70
## 0.976
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.272
## Median power for detecting 4/3PSM estimate.est
## 0.260Contour enhanced funnel plots
Forest plots
Moderator/sensitivity analyses
The below reported meta-regressions are all implemented as a multivariate RVE-based models using the CHE working model (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Testing of contrasts is carried out using a robust Wald-type test testing the equality of estimates across levels of the moderator.
Moderation by citations and IF
Overall effect moderated by citations and IF
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.4209 0.0307 13.70 <0.001 ***
## 2 scale(publicationYear) -0.0334 0.0293 -1.14 0.2535
## 3 scale(citationsGSMarch2016) 0.0626 0.0309 2.03 0.0424 *
## 4 scale(h5indexGSJournalMarch2016) -0.0880 0.0403 -2.18 0.0290 *
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI
## 1 intrcpt 0.4209 0.0307 Inf 0.36071
## 2 scale(publicationYear) -0.0334 0.0293 Inf -0.09081
## 3 scale(citationsGSMarch2016) 0.0626 0.0309 Inf 0.00214
## 4 scale(h5indexGSJournalMarch2016) -0.0880 0.0403 Inf -0.16688
## Upper 95% CI
## 1 0.48116
## 2 0.02395
## 3 0.12315
## 4 -0.00903Moderation by lattitude
Overall effect moderated by lattitude
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.4584 0.0367 12.482 <0.001 ***
## 2 scale(latitudeUniversity) 0.0267 0.0362 0.737 0.461
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.4584 0.0367 Inf 0.3864 0.5304
## 2 scale(latitudeUniversity) 0.0267 0.0362 Inf -0.0442 0.0975Compensatory effects moderated by lattitude
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.3051 0.0516 5.913 <0.001 ***
## 2 scale(latitudeUniversity) -0.0323 0.0528 -0.612 0.541
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.3051 0.0516 Inf 0.204 0.4063
## 2 scale(latitudeUniversity) -0.0323 0.0528 Inf -0.136 0.0712Priming effects moderated by lattitude
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.5120 0.037 13.86 <0.001 ***
## 2 scale(latitudeUniversity) 0.0451 0.037 1.22 0.223
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.5120 0.037 Inf 0.4396 0.584
## 2 scale(latitudeUniversity) 0.0451 0.037 Inf -0.0274 0.118Mood effects moderated by lattitude
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.232 0.0463 5.017 <0.001 ***
## 2 scale(latitudeUniversity) -0.016 0.0353 -0.451 0.652
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.232 0.0463 Inf 0.1417 0.3233
## 2 scale(latitudeUniversity) -0.016 0.0353 Inf -0.0852 0.0533Moderation by gender
Overall effect moderated by gender ratio
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.427 0.0338 12.63 < 0.001 ***
## 2 scale(percFemale) 0.103 0.0388 2.64 0.00821 **
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.427 0.0338 Inf 0.3609 0.494
## 2 scale(percFemale) 0.103 0.0388 Inf 0.0266 0.179Compensatory effects moderated by gender ratio
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.3062 0.0494 6.204 <0.001 ***
## 2 scale(percFemale) -0.0249 0.0568 -0.438 0.661
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.3062 0.0494 Inf 0.209 0.4030
## 2 scale(percFemale) -0.0249 0.0568 Inf -0.136 0.0864Priming effects moderated by gender ratio
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.467 0.0352 13.3 <0.001 ***
## 2 scale(percFemale) 0.148 0.0422 3.5 <0.001 ***
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.467 0.0352 Inf 0.398 0.536
## 2 scale(percFemale) 0.148 0.0422 Inf 0.065 0.231Published status
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 factor(published)0 0.312 0.0773 4.03 <0.001 ***
## 2 factor(published)1 0.432 0.0347 12.47 <0.001 ***
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(published)0 0.312 0.0773 Inf 0.160 0.463
## 2 factor(published)1 0.432 0.0347 Inf 0.364 0.500
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 2.05 1 24.3 0.164Comparing randomized and non-randomized designs
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 factor(rct)0 0.260 0.0543 4.79 <0.001 ***
## 2 factor(rct)1 0.447 0.0360 12.40 <0.001 ***
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(rct)0 0.260 0.0543 Inf 0.154 0.367
## 2 factor(rct)1 0.447 0.0360 Inf 0.376 0.517
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 8.72 1 34.7 0.00562 **Subgroups of observational and randomized effects
## k g [95% CI] SE Tau I^2 3PSM est [95% CI]
## Non.randomized 34 0.26 [0.14, 0.37] 0.06 0.22 80% 0.09 [-0.02, 0.21]
## Randomized 148 0.45 [0.38, 0.52] 0.04 0.29 70% 0.22 [0.1, 0.35]
## 3PSM.pvalue PET-PEESE est [95% CI] PET-PEESE.pvalue
## Non.randomized 0.117 -0.1 [-0.31, 0.12] 0.358
## Randomized 0.001 0.19 [0.04, 0.33] 0.011Non-randomized
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 29; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0387 0.1968 24 no study
## sigma^2.2 0.0116 0.1078 29 no study/result
##
## Test for Heterogeneity:
## Q(df = 28) = 102.6511, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2554 0.0567 4.5059 <.0001 0.1443 0.3665 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.255 0.0564 4.52 21.1 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.255 0.0564 21.1 0.138 0.373
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.223 0.734
##
## $Heterogeneity
## Tau I^2
## 0.2243704 79.6290061
## Jackson's I^2 Between-cluster heterogeneity
## 89.8100000 61.2600000
## Within-cluster heterogeneity ICC
## 18.3700000 0.7700000
##
## $`Proportion of significant results`
## [1] 0.3823529
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.4714771
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.092 0.059 1.569 0.117 -0.023 0.208 24.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.097 0.104 -0.938 0.358 -0.312
## ciUB PEESE estimate se zvalue pvalue
## 0.118 0.094 0.061 1.536 0.139
## ciLB ciUB
## -0.033 0.222
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.1493071 0.09129957 0.09129957 0.2436033 -0.2435232 0.5421375
## type kAdequate
## 1 1 3
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.14359084 0.01474893 0.27467682 0.03523204
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 23
## - Total number of p<0.05 studies included into the analysis: k = 11 (47.83%)
## - Total number of studies with p<0.025: k = 7 (30.43%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.274 -3.271 0.001 -3.501 0
## Flatness test 0.389 1.111 0.867 4.880 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 58% (22.5%-83.7%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.367"
## Median power for detecting a SESOI of d = .50
## "0.982"
## Median power for detecting a SESOI of d = .70
## "1"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.116"Randomized
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 144; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0464 0.2155 92 no study
## sigma^2.2 0.0364 0.1907 144 no study/result
##
## Test for Heterogeneity:
## Q(df = 143) = 559.4052, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4521 0.0364 12.4333 <.0001 0.3809 0.5234 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.452 0.0364 12.4 82.9 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.452 0.0364 82.9 0.38 0.525
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.124 1.028
##
## $Heterogeneity
## Tau I^2
## 0.2877887 69.8305082
## Jackson's I^2 Between-cluster heterogeneity
## 80.4600000 39.1500000
## Within-cluster heterogeneity ICC
## 30.6800000 0.5600000
##
## $`Proportion of significant results`
## [1] 0.6351351
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5386684
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.220 0.064 3.443 0.001 0.095 0.346 92.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.186 0.072 2.586 0.011 0.043
## ciUB PET estimate se zvalue pvalue
## 0.329 -0.145 0.096 -1.511 0.134
## ciLB ciUB
## -0.336 0.046
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 -0.02180643 0.1306439 0.1306439 0.8729201 -0.3414805 0.2978676
## type kAdequate
## 1 1 7
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.205756568 0.082334253 0.325994919 0.006398717
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 140
## - Total number of p<0.05 studies included into the analysis: k = 98 (70%)
## - Total number of studies with p<0.025: k = 63 (45%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.003 -8.542 0.00 -9.642 0
## Flatness test 0.076 1.887 0.97 12.097 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 47% (35.1%-58.7%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.204
## Median power for detecting a SESOI of d = .50
## 0.806
## Median power for detecting a SESOI of d = .70
## 0.977
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.183
## Median power for detecting 4/3PSM estimate.est
## 0.237F-test of equality of variances
Mean vi for non-randomized designs
## [1] 0.04069351Mean vi for randomized designs
## [1] 0.0660096F-statistics
## [1] 0.6164787F-test p-value
## [1] 0.1272321Comparing effects based on student and non-student samples
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 factor(studentSample)0 0.327 0.0456 7.16 <0.001 ***
## 2 factor(studentSample)1 0.453 0.0395 11.46 <0.001 ***
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(studentSample)0 0.327 0.0456 Inf 0.237 0.416
## 2 factor(studentSample)1 0.453 0.0395 Inf 0.375 0.530
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 4.36 1 81 0.0398 *Year of Publication
Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.
## Estimate Std. Error df t value
## (Intercept) -0.1407496 0.09437155 85.97906 -1.491442
## scale(h5indexGSJournalMarch2016) -0.1697075 0.10151050 85.90369 -1.671823
## scale(publicationYear) -0.2307599 0.08925779 85.99053 -2.585320
## Pr(>|t|)
## (Intercept) 0.13950560
## scale(h5indexGSJournalMarch2016) 0.09819827
## scale(publicationYear) 0.01141363Comment: all the variables were centered for easier interpretation of model coefficients. See the negative beta for Publication Year. The more recent a publication, the lower the variance (better precision), controlling for H5.
Scatterplot year <-> precision
Size of the points indicate the H5 index (the bigger the higher) of the journal that the ES is published in.
## `geom_smooth()` using formula 'y ~ x'
Citations
Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.
## Estimate Std. Error df t value
## (Intercept) -0.201059160 0.09182019 84.97688 -2.18970532
## scale(publicationYear) -0.003047723 0.11144001 85.08921 -0.02734855
## scale(h5indexGSJournalMarch2016) -0.365198106 0.11472691 84.73581 -3.18319482
## scale(citationsGSMarch2016) 0.332488656 0.10530882 85.10522 3.15727251
## Pr(>|t|)
## (Intercept) 0.031286371
## scale(publicationYear) 0.978245772
## scale(h5indexGSJournalMarch2016) 0.002037484
## scale(citationsGSMarch2016) 0.002203503Scatterplot precision <-> citations
The relationship between precision (sqrt of variance) and number of citations.
## `geom_smooth()` using formula 'y ~ x'
H5 index
Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.
## Estimate Std. Error df t value
## (Intercept) -0.1141591 0.0968350 87.04277 -1.178903
## scale(h5indexGSJournalMarch2016) -0.1178194 0.1027219 86.98377 -1.146974
## Pr(>|t|)
## (Intercept) 0.2416493
## scale(h5indexGSJournalMarch2016) 0.2545376Scatterplot precision <-> journal H5
The relationship between precision (sqrt of variance) and H5 index of the journal.
## `geom_smooth()` using formula 'y ~ x'
Decline effect
Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study.
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 0.001256607 0.08446073 86.44516 0.014878 0.98816379458
## scale(sqrt(vi)) 0.361986905 0.08744592 79.13890 4.139552 0.00008628732
## scale(publicationYear) 0.114376928 0.08487961 97.53930 1.347519 0.18093479180Citation bias
Do more highly-cited studies report larger effect sizes?
## Estimate Std. Error df t value
## (Intercept) 0.22906790 0.01409808 76.12334 16.2481604
## scale(publicationYear) -0.01667034 0.01740063 82.82924 -0.9580309
## scale(h5indexGSJournalMarch2016) -0.04319160 0.01701388 64.19127 -2.5386092
## scale(citationsGSMarch2016) 0.02599613 0.01651275 85.24009 1.5743067
## Pr(>|t|)
## (Intercept) 0.00000000000000000000000001855457
## scale(publicationYear) 0.34083527493986021106309181050165
## scale(h5indexGSJournalMarch2016) 0.01356484913606161198107447063421
## scale(citationsGSMarch2016) 0.11912106790969252678724643601527P-curve for interaction effects
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 51
## - Total number of p<0.05 studies included into the analysis: k = 37 (72.55%)
## - Total number of studies with p<0.025: k = 22 (43.14%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.162 -4.955 0.000 -6.205 0
## Flatness test 0.080 0.988 0.838 7.763 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 45% (25.8%-64.1%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: noP-curve and contour-enhanced funnel plots
Counts
Simple counts: 1. How often did authors test for moderation by attachment?
##
## 0 1
## 303 19- Via validated tests: How often was tested for:
- Independence of awareness
##
## 0
## 323- Lack of intention
##
## 0
## 320- Efficiency of behavior
##
## 0
## 317- Lack of control of behavior
##
## 0
## 321- Via non-validated tests: How often was tested for:
- Independence of awareness
##
## 0 1
## 0.7080925 0.2167630## [1] 51- Lack of intention
##
## 0
## 323- Efficiency of behavior
##
## 0
## 320- Lack of control of behavior
##
## 0
## 322- Whether researchers tested for innateness
##
## 0
## 320- Skin temperature location
##
## 1 2
## 6 5- Population type
##
## general special student
## 28 84 8 226- Control group type (1 = active controls)
##
## 0 1
## 6 37- In how many countries the studies were conducted and what their mean distance was from the equator
## [1] "USA" "" "China" "Portugal" "Singapore"
## [6] "Israel" "Germany" "South Korea" "Netherlands" "Japan"
## [11] "Scotland" "England" "India" "Canada" "Switzerland"
## [16] "Poland" "Italy"- Number of independent studies
## [1] 84- Number of papers
## [1] 33- Lattitude
## Lattitude mean Lattitude SD Min Max
## 39.74729 10.84424 1.29686 57.16498Further sensitivity analyses
Different step functions for the 3PSM
## steps deltaModerate deltaSevere deltaExtreme
## 1 0.0025 1.00 1.00 1.00
## 2 0.0050 0.99 0.99 0.98
## 3 0.0125 0.97 0.97 0.95
## 4 0.0250 0.95 0.95 0.90
## 5 0.0500 0.80 0.65 0.50
## 6 0.1000 0.60 0.40 0.20
## 7 0.2500 0.50 0.25 0.10
## 8 0.5000 0.50 0.25 0.10
## 9 1.0000 0.50 0.25 0.10## $deltaModerate
## est se zvalue pvalue ciLB ciUB k steps
## 0.320 0.032 10.115 0.000 0.258 0.383 114.000 9.000
##
## $deltaSevere
## est se zvalue pvalue ciLB ciUB k steps
## 0.236 0.036 6.613 0.000 0.166 0.306 116.000 9.000
##
## $deltaExtreme
## est se zvalue pvalue ciLB ciUB k steps
## 0.134 0.031 4.295 0.000 0.073 0.195 116.000 9.000## R version 4.0.2 (2020-06-22)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS 10.16
##
## Matrix products: default
## BLAS: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
##
## locale:
## [1] sk_SK.UTF-8/sk_SK.UTF-8/sk_SK.UTF-8/C/sk_SK.UTF-8/sk_SK.UTF-8
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] poibin_1.5 ddpcr_1.15 clubSandwich_0.5.2 weightr_2.0.2
## [5] scales_1.1.1 magrittr_2.0.1 multcomp_1.4-13 TH.data_1.0-10
## [9] MASS_7.3-51.6 survival_3.1-12 mvtnorm_1.1-1 Amelia_1.7.6
## [13] Rcpp_1.0.6 pwr_1.3-0 lmerTest_3.1-2 kableExtra_1.3.1
## [17] puniform_0.2.2 knitr_1.30 lme4_1.1-26 esc_0.5.1
## [21] dmetar_0.0.9000 meta_4.16-2 metafor_2.5-75 Matrix_1.2-18
## [25] psych_2.0.7 forcats_0.5.0 stringr_1.4.0 dplyr_1.0.1
## [29] purrr_0.3.4 readr_1.3.1 tidyr_1.1.1 tibble_3.0.6
## [33] ggplot2_3.3.3 tidyverse_1.3.0 reshape_0.8.8 car_3.0-9
## [37] carData_3.0-4
##
## loaded via a namespace (and not attached):
## [1] minqa_1.2.4 colorspace_2.0-0 ellipsis_0.3.1
## [4] class_7.3-17 modeltools_0.2-23 rio_0.5.16
## [7] mclust_5.4.7 fs_1.5.0 rstudioapi_0.13
## [10] farver_2.0.3 ggrepel_0.9.1 flexmix_2.3-17
## [13] lubridate_1.7.9 mathjaxr_1.2-0 xml2_1.3.2
## [16] codetools_0.2-16 splines_4.0.2 mnormt_2.0.1
## [19] robustbase_0.93-7 jsonlite_1.7.2 nloptr_1.2.2.2
## [22] broom_0.7.0 cluster_2.1.0 kernlab_0.9-29
## [25] dbplyr_1.4.4 compiler_4.0.2 httr_1.4.2
## [28] backports_1.1.8 assertthat_0.2.1 cli_2.3.0
## [31] htmltools_0.5.0 tools_4.0.2 gtable_0.3.0
## [34] glue_1.4.2 cellranger_1.1.0 vctrs_0.3.6
## [37] nlme_3.1-148 fpc_2.2-9 xfun_0.19
## [40] openxlsx_4.1.5 rvest_0.3.6 CompQuadForm_1.4.3
## [43] lifecycle_0.2.0 statmod_1.4.35 DEoptimR_1.0-8
## [46] zoo_1.8-8 hms_0.5.3 sandwich_2.5-1
## [49] parallel_4.0.2 yaml_2.2.1 curl_4.3
## [52] gridExtra_2.3 MuMIn_1.43.17 stringi_1.5.3
## [55] highr_0.8 boot_1.3-25 zip_2.1.0
## [58] rlang_0.4.10 pkgconfig_2.0.3 prabclus_2.3-2
## [61] evaluate_0.14 lattice_0.20-41 labeling_0.4.2
## [64] tidyselect_1.1.0 plyr_1.8.6 R6_2.5.0
## [67] generics_0.0.2 DBI_1.1.0 mgcv_1.8-31
## [70] pillar_1.4.7 haven_2.3.1 foreign_0.8-80
## [73] withr_2.4.1 abind_1.4-5 nnet_7.3-14
## [76] modelr_0.1.8 crayon_1.4.0 tmvnsim_1.0-2
## [79] rmarkdown_2.5.3 grid_4.0.2 readxl_1.3.1
## [82] data.table_1.13.0 netmeta_1.3-0 blob_1.2.1
## [85] webshot_0.5.2 reprex_0.3.0 digest_0.6.27
## [88] diptest_0.75-7 numDeriv_2016.8-1.1 stats4_4.0.2
## [91] munsell_0.5.0 viridisLite_0.3.0 magic_1.5-9